Definition and Validation of Requirements Volatility Measures

This paper is a summary of a PhD thesis which describes our effort to define and validate requirements volatility measures. These measures have been proven to be reliable predictors of requirements volatility. 1. TECHNICAL PROBLEM Since requirements often change, even during development, it is important to predict the continuing definition of requirements as they change throughout the software life cycle to be able to anticipate and respond to requests of change. The consequences of poor management of changes would be higher project costs, risks for schedule slippage, and decrease in quality. These consequences could lead to project failure. Even though there are high risks in managing requirements, this is in general not done properly. In a survey of 4000 European companies it was found that the management of customer requirements was one of the main problem areas in software development. Among the requirements management (RM) activities, measuring and predicting volatility is very important. High volatility can cause cost and schedule overruns, making the goals of the project hard to achieve. Numerous software measures for the RM activities and for requirements volatility have been proposed in the literature. To our knowledge, none of them have been validated. The goal of this research is therefore to define and validate requirements measures, proving that they can be used as reliable predictors of requirements volatility. 2. CONTRIBUTIONS AND METHODS USED We defined a general and wide set of 38 measures for the management of requirements [3]. The measures were obtained by applying the Goal Question Metrics paradigm to the RM Key Process Area of the Capability Maturity Model for software. This set is not exhaustive, other measures can be defined. It constitutes a pick list that can be tailored to the specific company, offering small-medium enterprises the freedom to choose a suitable subset of software measures. The set defined differs from others because our measures are focused on a specific process area. Measures are not useful if their practical utility is not demonstrated empirically. Therefore we need to validate the measures defined. Measures validation requires a convincing demonstration that: 1. the measure measures what it purports to measure (theoretical validation); 2. the measure is associated with an important external quality attribute (empirical validation). Ten of the 38 measures have been theoretically validated in [4], by applying two validation procedures [1], [2] (table 1 shows two validated measures). For the empirical validation, we performed two academic case studies [4], [5], where we used the measures defined. As a result, we learned that: 1) it is important to keep track of change requests status; 2) the amount of data to be collected should be kept small; 3) effort should be spent in pushing developers and stakeholders to discuss their requirements. An industrial case study described in [6] was performed with two goals: 1) to empirically validate a set of measures associated with the volatility of use case models (UCM); 2) to investigate the accuracy of subjective estimations in predicting requirements volatility. Measurement data was collected in retrospect for all use case models of the software project. In addition, we determined subjective volatility by interviewing stakeholders of the project. The data analysis showed a high correlation between our measures of size of UCM and total number of changes, indicating that the measures of size of UCMs are good indicators of requirements volatility. On the other hand, subjective estimations were not found to be good indicators of volatility. Based on the results from [6], we are currently performing a correlational study in order to construct and validate prediction models of requirements volatility. The models are based on data collected from two industrial projects for five measures of requirements size (number of lines, words, actors, use cases, and revisions). Applying univariate and multivariate regression analysis, we built prediction models using data collected on a medium size software project. We then evaluated the models accuracy by applying the models on a set of data collected on a second, slightly larger, project developed at the same company. Preliminary results of the study show that the measures based on Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SERPS'06, October 18-19, 2006, Umeå, Sweden requirements length are the best predictors of volatility. In fact, as we can observe in table 2, model two is the most accurate. It performs better than COCOMO which has MMRE=0.6 and Pred(0.25)=0.27. Other prediction models based on measures of complexity and functionality were found less accurate.